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Brain is the most important part of the nervous system. Brain tumor is mainly 
a mass or growth of abnormal tissues in a brain. Early detection of brain tumor 
can reduce complex treatment process. Magnetic resonance images (MRI) are 
used to detect brain tumor. In this paper, we have introduced a deep 
convolutional neural network (CNN) to automatic brain tumor segmentation 
using MRI medical images which can solve the vanishing gradient problem. 
Classifying the brain MRI images with Resnet-50 and InceptionV3 in order to 
identify whether there is tumor or not. After this step, we have compared the 
accuracy level of both of the CNN models. Thereafter, applied U-Net 
architecture individually with encoder Resnet-50 and InceptionV3 to avieved 
promising results. The publicly available low-grade gliomas (LGG) 
segmentation dataset has been utilized to test the model. Before applying the 
model on the MRI images preprocessing and several augmentation techniques 


Segmentation have been done to obtain quality a dataset. U-net architecture with 
U-Net InceptionV3 provided 99.55% accuracy. On the other hand, our proposed 
method U-net with encoder ResNet-50 showed 99.77% accuracy. 
This is an open access article under the CC BY-SA license. 
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1. INTRODUCTION 

A brain tumor is the growth of abnormal cells in a human brain. The tumor first attacks a part of the 
brain cells and then slowly spreads throughout the cells. After attacking the brain cell, the brain cell is damaged. 
Brain tumor can be cancerous or non- cancerous. Benign is the non-cancerous and malignant is the cancerous 
which are the type of brain tumors. A primary brain tumor begins in brain cells. on the other hand, secondary 
brain tumor cancer cell spread into brain from another body part such as lung, breast. Gliomas is called of 
integral tumor [1]. Basically, gliomas is a brain tumor which can be graded into low grade gliomas (LGG) and 
high grade gliomas (HGG) [2]. In the recent time most of the patients suffered from HGG. When a patient is 
diagnosed with HGG, their life is at risk. The patients diagnosed with HGG has lower life expectancy than the 
patients diagnosed with LGG [3]. 

Brain tumors make up 1.8% of all the cancer incidents worldwide. The early detection of brain tumors 
and the correct type of cancer would help doctors in selecting the proper treatments and in further analysis 
based on how the patients responded to the treatment [4]. Brain tumors are treated based on the age of patients, 
the type of tumors and the location of tumors. Pictures of brain tumors are captured with magnetic resonance 
imaging (MRI). It is a difficult work to segment a brain tumor to diagnose by MRI for complex structures and 
it is time consuming. It is difficult to diagnose damaged tissue from healthy tissue because of the appearance 
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of the tumor, blurred boundaries. We can use automatic brain tumor segmentation in order to solve this problem 
using MRI images which can identify the type of tumor and the exact location of the effected cell [5]. Image 
segmentation can be partitioned from multiple objects/segments to a single object. It performs labeling of pixel- 
level for all image pixels which predicts a single label for the whole image. [6]. Rai et al. [7], proposed transfer 
learning approach where brain tumors can be graded by combining U-net and ResNet-50. Cinar and Yildirim 
used InceptionV3 where they removed the last 5 layers. By using this architecture, they got 97.2% accuracy 
because of adding eight new layers [8]. 

Yusuf Artan combined two of the successful segmentation algorithms named the limited discrepancy 
search (LDS) and the random walker (RW) and tested the effectiveness of the method from the grabcut and 
berkeley segmentation database and also was compared with the Raw Walker algorithm to assess the 
performance [9]. Portela et al. [10] proposed a method where human specialized inspection is not required nor 
set of labeled training dataset. Nandi [11] used MRI images as the normal MR image analysis are not suitable 
for the analysis. K-means clustering was used where some abnormality was shown by the detected tumor. 

Pereira et al. [12] proposed an automated segmentation method on convolutional neural networks 
(CNN), applied on Brats2013 and 2015 datasets. It was done by creating and fine tuning each tumor grade’s 
intensity normalization transformation. Manogaran et al. [13] proposed a method which used an approach of 
upgraded orthogonal gamma distribution-based machine learning. With automatic region of interest (ROD 
detection the area of brain tumor was calculated. Siar and Teshnehlab [14] detected brain tumor from the 
magnetic resonance images also known as MRI. They used convolutional neural network also known as CNN. 
The softmax, radial basis function and decision tree (DT) were used to classify the images. Two class classifier 
is implemented using support vector machine with radial bass function (SVM-RBF) kernel. The proposed 
method can achieve an accuracy of >94% [15]. Khan et al. [16] used deep learning on brats 2015, 2017, and 
2018 and claimed a new method called extreme machine learning (ELM). 

In recent years the brain tumor detection was way better due to some advanced technologies. An 
automatic MRI brain tumor classification has been presented by Kumar ef al. [17]. Preprocessing, feature 
extraction, classification and segmentation has been done on brain tumor segmentation (BRATS) and Miccai 
datasets. The images were converted into 3x3 blocks. Rehman et al. [18] presented an encoder-decoder based 
model. The model mainly used FE blocks at every encoder stage. Also, the model gets a feature map and 
defined operation is performed by the model which also tries to preserve information. This feature of aggression 
helps a lot to get better execution of finding brain tumors. Brain tumors were classified using MRI data analysis 
in order to help the practitioners. In order to do that they used deep learning methods. VGG19 with k-means 
cluster was used here [19]. Ligiang et al. [20] developed a unique network fine-tuning technique based on 
policy value. The datasets BraTS 2016, BraTS 2017, and BraTS 2018 were utilized in their methodology. They 
also used a number of experiments to show that their proposed strategy may be used in the medical field. 

This paper focused on image segmentation for classifying tumor from brain MRI images and 
identifying the exact location of brain tumors. U-Net is used for medical image segmentation. Figure | presents 
the overall concept of our proposed study. The proposed method is a combination of U-net architecture with 
the encoder ResNet-50. Preprocessing is one of the most important techniques which can be improved image 
quality before applying method. Classification has been done with ResNet-50 and Inception-V3 which shown 
that ResNet-50 has given promising result. This may allow for the diagnosis of a brain tumor at an earlier stage. 
The medical community can help lower the incidence of brain tumors by improving their ability to detect them 
early. Brain tumors are another possible route for the spread of brain cancer. The sooner the brain tumor is 
diagnosed, the sooner therapy can begin. Which ultimately leads to a lower incidence of brain cancer. 
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Figure 1. A complete system architecture to implement an android application using proposed model 
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2. DESCRIPTION OF THE PROCEDURE 

In the proposed model we combined the U-net architecture with ResNet-50 encoder. Here, we applied 
these approaches to get better accuracy in brain tumor segmentation. The whole process of the proposed work 
has been done as following steps which is illustrated in Figure 2. 
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Figure 2. Flow chart of work process 


2.1. Data collection and pre-processing 

LGG segmentation dataset was used to evaluate the proposed model. Brain MR images together with 
manual fluid-attenuated inversion recovery (FLAIR) abnormality segmentation masks are included in the 
dataset. The cancer imaging archive (TCIA) is where the MR images were collected. TCGA’s lower-grade 
glioma collection includes 110 patients with FLAIR sequence and genomic cluster data available from the 
TCGA lower grade glioma collection. The dataset contains total of 7,858 where 3,928 are MRI Images and 
3,929 are segmentation masks. Figure 3 to illustrate a sample of dataset after resize the images into 250x250 
frame. 


Figure 3. Brain MRI Images without tumor 
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Pre-processing on an image data often improve its quality and clarity. Different preprocessing 
techniques can be used to achieve desired results. A pre-processing step is done on the MRI images before they 
are fed into the proposed structure [21]. It is used to generate batches of tensor images with real-time data 
augmentation in the image preparation component of the keras library. ImageDataGenerator method has the 
preprocess input parameter set to the appropriate MRI image format for the model. 


2.2. Data augmentation 

The success of deep learning (DL) models generally depends on the size of the dataset they are trained 
on. However, by augmenting the existing data, we can enhance the model’s performance. We used 
ImageDataGenerator in order to do data augmentation. ImageDataGenerator is mainly a real-time, batch-based 
tensor image data generator. A data loop is performed (in batches). As a result, it can instantly apply random 
changes to a group of images. Each training iteration uses a slightly different collection of input images thanks 
to the random changes done at the start of each epoch. 

In this study, several data augmentation methods, like shifting, rotation, flipping and zooming have 
been used to expand dataset. We zoomed +5% to the original dataset to observe the tumor position clearly. We 
also applied the horizontal and vertical shift both +10% to gain the sufficient training dataset from the original 
dataset. Horizontal flip and shear +15% on both horizontal and vertical direction have also been applied to gain 
the powerful training dataset. 


3. PROPOSED MODEL 

Two CNN models ResNet-50 and InceptionV3, were utilized to classify the MRI images. Proposed 
network was then chosen based on their performance. For images assessed as nontumorous, the algorithm 
terminates, whereas images classified as tumorous are sent to the next phase of the architecture. 


3.1. Introduction to InceptionV3 

In 2014 Google introduced a network which was a pre-trained network model also known as Google 
net [22]. With 22 layers, 5M parameters, and filter sizes of 1x1, 3x3, and 5x5, Inception was able to extract 
features at various scales coupled with maximum pooling. 1x1 filters are used to speed up computations. 
InceptionV3 was introduced by google in 2015 upgrading the inception model [23]. In the InceptionV3 model 
factorization of convolutional layers is used to reduce the number of parameters. Using two 33 filters instead 
of the 5x5 Convolutional filters to reduce processing without affecting network speed. The architecture of 
Brain tumor classification with Inception v3 is given in Figure 4. 
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Figure 4. Brain tumor classification with InceptionV3 


3.2. Introduction to ResNet-50 

The structure of deep learning-based network is getting deeper as it is evolving [24]. However, it can 
cause new problem such as Vanishing gradient problem during the training process. Generally, it takes more 
power of computational in order to train the deep neural network [25]. ResNet50 is 50 layers deep networks 
and which can classify pictures into 1000 categories. Skip connection is a technique which is used in Residual 
networks. By skipping some layers in between, the ResNet tends to connect activations of a layer to layers 
which are far. As a result, a residual block is created. ResNets are then made by stacking these blocks with one 
another. Instead of the underlying layers be learned by the layers the residual mapping is fit by the network, 
instead of H(x) in (1), initial mapping the network fit. 
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The benefit of using this kind of connection skipping is that the layer will be skipped if it hurt the 


execution of the architecture by the regularization. As a result, the vanishing gradient problem can be resolved. 
The architecture of brain tumor classification learning framework of ResNet is shown Figure 5. 
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Figure 5. Brain tumor classification with ResNet-50 


3.3. Image segmentation with U-Net 

It has been a remarkable revolution in the field of deep learning after the invention of U-net. First 
used for biomedical image segmentation, U-net was originally developed and first deployed. A decoder 
network follows an encoder network in this system's design. Semantic segmentation is different from 
classification in that it necessitates both pixel-level discrimination and the ability to project discriminative 
features learned at different stages of the process onto the pixel space. 

We utilized a ResNet-50 encoder and convolution blocks followed by max-pooling and down- 
sampling to encode the input image into feature representations. The architecture is complete when the decoder 
is included. To accomplish a high level of classification, the encoder must learn how to correctly project 
discriminative features from lower resolutions into the higher resolution images. The decoder consists of up- 
sampling and splicing, followed by conventional procedures. The following Figure 6 shows the prediction 
result of proposed model, here we selected U-net with ResNet-50 as our proposed model. 
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4. EXPERIMENTAL RESULTS AND DISCUSSION 

Our dataset contains brain MRI images of 110 patients and data is split into training, validation and 
testing purpose. Pre-processing the dataset and then experiment has been executed on a CPU having 4.10 GHz 
core i7 processor with 16 GB of ram. 10 epochs with batch size of 16 our models have been trained. Brain 
tumor classification has been done with two CNN models: ResNet-50 and Inception v3. To increase the 
performance, we combined U-net with ResNet-50 and Inception v3 and we observed U-Net with ResNet-50 
generated the best result on our dataset. 


Table 1. Model’s performance according to different measurement units 


Model Accuracy Precision Score __ Recall _ Fl-Score 
ResNet-50 0.9929 0.9869 0.9927 0.9898 
Inception V3 0.9946 0.9898 0.9949 0.9923 
Unet + Inception V3 0.9955 0.9912 0.9956 0.9934 
Unet + ResNet50 0.9977 0.9956 0.9978 0.9967 


The calculation of actual true classification is called accuracy. The estimation of predicted positive 
labels is called Precision. How many positive labels was correctly predicted from our dataset is called Recall 
and the weighted mean of Recall and Precision is the Fl-score. Table 1 shows the Accuracy, Precision, Recall 
and the Fl-score of applied models. These values calculated from confusion matrix, shows in Figure 7. Here, 
Figure 7(a) presents Unet and ResNet50, and Figure 7(b) shows Unet and Inception V3 which is easily help us 
to find out required performance units. For comparing the proposed model, CNN based networks were listed 
here. Table 2 reports the overall comparison results. Figure 8 shows the accuracy level on training and 
validation dataset of the proposed model according to epochs. Here, the diagram present two networks with U- 
Net segmentation approach. It is obvious that, image segmentation emphasis model’s performances. 
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Figure 7. Confusion matrix; (a) Unet + ResNet50 and (b) Unet + Inception V3 
Table 2. CNN based model’s comparison for brain tumor prediction 
Reference CNN Model Pre-processing Segmentation Dataset Accuracy 
Siar and CNN+Softmax N/A Alexnet Personal 98.67% 
Teshnehlab [14] CNN+RBF 97.34% 
CNN+DT 94.24% 
Clustering and 99.12% 
CNN+Softmax 
Pugalenthi et al. N/A SGO+FTT Level-Set (LSS), BRATS2015 94.33% 
(15] GLCM Feature 
extraction 
Khan et al. [16] VGG16 Linear Contrast Enhancement N/A BraTs2015 97.8% 
VGG19 BraTs2017 96.9% 
BraTs2018 92.5% 
Rehman et al. VGG19 SimpleITK, k-means clustering BraTS 2015 97.8% 
[18] Generation of brain pipeline, BraTS 2017 96.9% 
normalization. BraTS 2018 92.5% 
Proposed. ResNet-50 ImageDataGenerator of Keras Unet+ResNet-50 LGG segmented 99.77% 
library dataset 
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Figure 8. Accuracy curve of brain tumor prediction 


5. CONCLUSION 

Early brain tumor detection can increase the rate of survival of a patient. magnetic resonance imaging 
(MRI) has become very popular in terms of brain tumor detection. But sometimes it can’t be very reliable. 
Image segmentation can be used in order to detect brain tumor more sufficiently. In this work a U-net 
architecture introduced with Resnet-50 encoder and Inception v3. Which was tested on the LGG segmentation 
dataset. However, we pre-processed our dataset using several augmentation techniques. The U-net with ResNet 
-50 able to solve the vanishing gradient problem with tremendous accuracy of 99.77%. In future, we will 
increase the size of data by collecting more brain MRI images from different sources. It is also possible to 
apply our proposed model on different publicly available datasets like Brats. In future, we will collect live data 
from different hospital and clinic to enrich our research work. For gaining the highest classification accuracy 
we may utilize genetic algorithm, simulated annealing, particle swarm optimization and other different kind of 
optimizing algorithms. 
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